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PREFACE 


This  document  contains  a  summary  of  each  Air  Force  Human  Resources  Laboratory  Technical 
Report  and  Technical  Paper  that  the  Human  Systems  Division  Public  Affairs  Office  has 
approved  for  public  release  during  the  past  3  months.-  It  does  not  include  summaries  of 
Special  Reports,  nor  Technical  Reports  and  Technical  Papers  not  cleared  for  public 
distribution.  -r  j  /»  ■■■’,  .  -  <•  \ 

...  i  '  ~ 

In  addition,  the  unlimited  distribution  technical  documents  published  during  this  j 

quarter  are  listed.  All  are  available  from  the  Defense  Technical  Information  Center, 

Cameron  Station,  Alexandria,  Virginia  22314,  to  qualified  requesters  and  from  the 
National  Technical  Information  Service,  Springfield,  Virginia  22151.  The  AD  number 
should  be  included  on  requests  for  documents. 


SUMlARIES 
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Mumford,  M.D.,  Haraing,  F.D.,  Fleishman,  £.A.,  4  Weeks,  J.L.^An  empirical  system  for  assessing 
the  impact  of  aptitude  requirement  adjustments  on  Air  Force  initial  skills  training ' 
( AFHRL-TR-86-1 9 ) .  Brooks  AFB,  TX:  Manpower  and  Personnel  Division. 

The  objective  of  this  research  and  development  (R4D)  effort  was  to  develop  ana  validate  an 
empirical  procedure  for  predicting  training  outcomes  associated  with  initial-skills  courses 
administered  by  the  United  States  Air  Force  Air  Training  Command.  The  R4D  consisted  of  the 
collection  and  analysis  of  information  to  determine  the  effects  of  student  attributes  and  course 
properties  on  training  outcomes  such  as  academic  training  performance  and  student  elimination. 

The  value  of  the  procedure  resides  in  its  use  as  a  decision  support  system  for  personnel  and 
training  management.  It  can  be  applied  on  a  course-by-course  basis  to  evaluate  "what-if" 
questions  concerning  the  effects  of  changes  in  student  ana  course  characteristics  on  training 
outcomes.  For  example,  the  system  could  be  used  by  personnel  managers  to  forecast  the  training 

outcomes  likely  to  result  from  decreases  in  the  minimum  aptitude  test  score  required  for  entry 

into  an  ini tial -skil 1 s  course.  It  could  be  used  by  training  managers  to  forecast  the  training 

outcomes  likely  to  result  from  increases  or  decreases  in  the  trained  personnel  requirement  for 
the  course.  It  could  be  used  by  course  designers  to  forecast  the  training  outcomes  likely  to 
result  from  adjustments  in  course  length  or  student/i nstructor  ratio.  In  addition  to  providing  a 
means  of  forecasting  training  outcomes  associated  with  changes  in  single  variables,  it  could  be 
used  to  forecast  training  outcomes  associated  with  simultaneous  adjustments  in  several  student 
and  course  characteristics.  The  payoff  of  this  R4D  is  a  decision  support  system  for  personnel 
and  training  management  that  can  be  applied  to  anticipate  ano  avoid  adverse  training  outcomes. 


Drasgow,  F.,  Levine,  M.V.,  McLaughlin,  M.E.,  4  Earles,  J.A.  Appropriateness  measurement 

( AFHRL-TP-87-6,  AD-A184  185).  Brooks  AFB,  TX:  Manpower  and  Personnel  Division. 

The  military  services  have  a  vital  concern  in  assuring  that  aptitude  test  scores  are 
appropriate  measures  of  examinees'  true  abilities.  Substantial  bonuses  have  been  paid  to 
examinees  with  sufficiently  high  scores  as  enticement  to  enlist  into  selected  occupations.  Under 
mobilization,  exemption  from  service  will  be  given  to  examinees  with  unacceptably  low  scores. 
Therefore,  cheating  to  Improve  scores  and  deliberately  picking  Incorrect  answers  to  lower  scores 
are  both  plausible  threats  to  the  integrity  of  enlistment  testing.  The  goal  of  Appropriateness 
Measurement  is  to  develop  ways  to  analyze  examinees'  responses  to  multiple-choice  tests  so  as  to 
identify  such  inappropriate  test  responding. 

This  effort  evaluates  11  practical  appropriateness  indices.  Three,  which  are  based  on  modern 
test  theory  (Item  Response  Theory),  were  found  to  effectively  detect  aberrant  response  patterns 
across  a  fairly  wide  range  of  conditions.  This  success  was  obtained  when  the  test  had  many  items 
but  was  substantially  lessened  for  military  selection  test  lengths.  However,  methods  developed 
for  combining  information  on  aberrant  responding  across  several  different  tests  resulted  in  an 
effectiveness  comparable  to  that  found  with  the  longer  tests. 

The  results  strongly  suggest  that  appropriateness  indices  can  be  used  effectively  in 
operational  settings.  Further  research  Is  suggested  on  a  class  of  indices  called  "optimal"  which 
hold  the  promise  of  even  better  Identification  of  aberrant  responding  than  those  Indices  already 
identified. 
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Hedge,  J.W.,  &  Lipscomb,  M.S.  Walk-through  performance  testing:  An  innovative  approach  to  work 
sample  testing  ( AFHRL-TP-87-8) .  Brooks  AFB,  TX:  Training  Systems  Division. 


As  part  of  an  extensive  job  performance  measurement  research  and  development  program,  the  Air 
Force  Human  Resources  Laboratory  has  developed  a  new  methodology  cal  lea  Walk-Through  Performance 
Testing  (WTPT).  wTPT  is  a  task-level  job  performance  measurement  system  which  combines  hands-on 
testing  and  interview  testing  to  proviae  a  high-fidelity  measure  of  an  inaiviaual's  technical  job 
competency.  This  document  contains  a  series  of  papers,  originally  presented  at  the  92na 
Convention  of  the  American  Psychological  Association,  which  outline:  (a)  the  conceptual  frame- 
of-reference  within  which  the  original  planning  for  WTPT  took  place,  (b)  the  WTPT  methodology  and 
the  rationale  and  overall  approach  to  hands-on  and  interview  test  development,  (c)  the  sampling 
strategy  used  to  select  tasks  for  work  sample  development,  and  (d)  the  approach  used  to  analyze 
selected  tasks  for  WTPT  development.  A  final  section  discusses  the  implications  of  the 
measurement  strategy. 


Kavanagh,  M.J.,  Borman,  W.C.,  Hedge,  J.W.,  &  Gould,  R.B.  Job  performance  measurement  in  the 
military:  A  classification  scheme,  literature  review,  and  directions  for  research 
(AFWL-TR-87-15).  Brooks  AFB,  TX;  Training  Systems  Division.  ' 

The  job  performance  measurement  literature  indicates  that  previous  research  relied  heavily  on 
broad-based  generic  indices,  performance  ratings,  or  operational  measures  with  their  inherent 
problems  of  Inflation  and  halo  effects.  These  broad  measures  were  unable  to  take  into  account 
task-level-specific  Influences  such  as  training  differences  or  opportunities  to  perform;  hence, 
such  efforts  have  been  largely  unsuccessful.  However,  it  appears  that  current  interest, 
resources,  and  state-of-the-art  technology  developments  have  now  significantly  increased  the 
probability  of  developing  successful  measures  of  job  performance.  This  report  describes  the  Air 
Force  Human  Resources  Laboratory's  (AFWL)  research  program  for  development  of  individual  Job 
performance  measures.  The  report  describes  the  construction  of  a  job  performance  measurement 
classification  scheme  into  which  the  relevant  empirical  and  theoretical  literature  are 
organized.  Based  on  this  framework,  specific  recommendations  for  both  applications  and  research 
directions  are  given. 
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Branat,  C.M.,  Jernigan,  J.H.,  i  Dierker,  R.J.  Integrateo  Maintenance  Information  System: 
Training  technology  scenarios,  (AFHRL-TP-87-18).  Wright-Patterson  AFB,  OH:  Logistics  and  Human 
Factors  Division. 

The  Air  Force  Human  Resources  Laboratory  (AFHRl)  is  developing  an  Integrated  Maintenance 
Information  System  <IMIS).  IMIS  will  demonstrate  the  capability  to  access  and  integrate 
information  from  multiple  sources  and  present  the  information  to  technicians  through  a  rugged, 
hand-held  computer.  Results  of  the  program  will  form  the  basis  of  requirement  specifications  for 
such  a  system. 

AFHRL  is  performing  preliminary  research  in  many  areas  key  to  the  success  of  the  IMIS 
concept.  These  areas  include  digitia?  authoring  and  presentation  of  maintenance  instructions, 
integrated  diagnostics  aiding,  human-computer  interaction,  advanced  portable  computer  hardware 
technologies,  and  potential  training  applications.  IMIS  will  be  the  culmination  of  a  complex  and 
thorough  research  and  development  effort.  Specifications  developed  by  the  project  will  be 
validated  through  field  tests  performed  by  Air  Force  maintenance  technicians.  As  a  result,  IMIS 
will  Improve  the  capabilities  of  maintenance  organizations  to  effectively  utilize  available 
manpower  and  resources  to  meet  combat  sortie  generation  requirements. 

In  the  work  being  reported,  representative  base-level  maintenance  training  requirements  were 
identified.  Based  on  these  requirements,  a  series  of  "operational  scenarios"  were  developed  to 
help  conceptualize  how  IMIS  technology  might  be  used  to  support  base-level  maintenance  training. 
Based  on  this  work,  it  appears  that  IMIS  can  improve  upon  present  methods  used  to  provide  this 
training. 
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Stoker,  P.,  Hunter,  O.R.,  Batchelor,  C.L.,  &  Curran,  L.T.  Air  traffic  controller  trainee 
selection  ;(AFHRL-TP -87-19).  Brooks  AFB,  TX:  Manpower  and  Personnel  Division. 

The  objective  of  this  effort  was  the  development  of  an  improved  method  for  the  selection  of 
first-term  personnel  for  entry  into  air  traffic  controller  training. 

Rising  attrition  levels  during  both  training  and  initial  assignments  led  the  Air  Force 
Communications  Command  to  issue  a  Request  for  Personnel  Research  to  reevaluate  existing  selection 
measures  and  develop  new  measures,  if  neeoeo.  At  that  time,  the  General  Aptituae  Index  (AI)  and 
Administrative  AI  from  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  were  used  for 
selection. 

First,  the  relationships  between  ASVAB  AIs  and  training  performance  were  a ssesseo.  In 
several  independent  samples,  witn  a  total  of  over  3,000  subjects,  it  was  found  that  the  General 
AI,  Mechanical  AI,  and  Electronics  AI  correlated  well  with  training  performance,  whereas  the 
Administrative  AI  had  a  smaller  relationship  with  the  criteria.  On  the  basis  of  these  analyses, 
the  Air  Force  Human  Resources  Laboratory  recommended  the  deletion  of  the  Administrative  AI  as  a 
selector  for  entry  into  air  traffic  controller  training.  That  recommendation  was  accepted  and 
the  use  of  the  Administrative  AI  was  discontinued. 

In  addition,  five  tests  were  selected  for  experimental  validation:  (a)  Multiplex  Controller 
Aptitude  Test  (MCAT),  (b)  Object  Completion  Test  (OCT),  (c)  Rotated  Blocks  Test  (RBT) ,  (d) 

Perceptual  Abilities  Test  (PAT),  and  (e)  Electrical  Maze  Test  (EMT).  These  five  tests  were 

administered  to  457  air  traffic  controller  candidates  prior  to  the  start  of  training.  Test 
scores  were  compared  to  a  dichotomous  training  criterion— pass  versus  failure  in  training.  All 
tests,  except  the  PAT,  were  significantly  correlated  with  the  pass  versus  fail  criterion.  The 
MCAT  was  the  best  single  predictor.  Multiple  regression  analyses  showed  that  the  combination  of 
MCAT  and  RBT  yielded  the  best  combined  prediction,  and  that  their  use  would  Improve  upon 

prediction  made  by  using  the  ASVAB  alone. 

When  evaluated  against  first-year  job  success/attrition,  the  AIs  were  not  found  to  relate 

significantly  to  this  criterion;  only  the  PAT  related  significantly  to  the  criterion. 
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Carretta,  T.R.  Basic  Attributes  Test  (BAT)  system:  A  preliminary  evaluation  (AFHRL-TP-87-kO). 
Brooks  AFB,  TX:  Manpower  and  Personnel  Division. 


High  rates  of  attrition  among  students  In  Undergraduate  Pilot  Training  (UPT)  are  a  major 
concern  for  the  Air  Force.  Recent  research  and  development  efforts  at  the  Air  Force  Human 
Resources  Laboratory  have  attempted  to  reduce  attrition  rates  by  improving  the  method  by  which 
pilot  candidates  are  selected.  Currently,  UPT  students  are  chosen  primarily  on  the  basis  of 
their  scores  on  the  Pilot  composite  of  the  Air  Force  Officer  Qualifying  Test  (AFOQT).  The 
present  effort  sought  to  determine  the  extent  to  which  scores  on  three  cognitive/perceptual 
subtests  from  an  experimental  test  battery,  known  as  the  Basic  Attributes  Test  (BAT),  aadeo  to 
the  validity  provided  by  the  AFOQT  Pilot  composite  score. 


Scores  from  the  three  cognitive/perceptual  tests— Digit  Memory  (information  input 
efficiency),  Decision-Making  Speed  (choice  reaction  time),  and  Item  Recognition  (short-term 
memory  storage,  search  and  comparison  operations)— did  not  add  significantly  to  the  prediction  of 
graduation  or  failure.  However,  the  experimental  subtests  did  demonstrate  significant 
relationships  with  several  other  performance  measures  Including  recommendations  for  fighter  or 
non-fighter  assignments  following  UPT. 


Polzella,  3.-.,  “ubbara,  D.i..,  Brown,  J.E.,  4  McLean,  H.C.  Aircrew  training  devices:  Utility 
an  a  utilization  of  ddvancea  instructional  features ,( Phase  IV  -  Summary  report)  (AFHRL-TR-67-21 ) . 
williams  AFB ,  AZ :  Operations  Training  Division. 

Modern  Aircrew  Training  Devices  (ATDsl  are  equipped  with  sophisticated  hardware  and  software 
capacil  i  ties,  known  as  Advanced  Instructional  Features  (AIFs),  that  permit  a  simulator  instructor 
(SI)  to  prepare  briefings,  manage  training,  vary  task  difficulty/fidelity,  monitor  performance, 
and  provide  feedback  for  flight  simulation  training  missions.  The  utility  and  utilization  of  the 
AIF  capabilities  of  USAF  ATDs  was  explored  by  means  of  a  survey  of  534  Sis  from  Air  Training 
Command,  Military  Airlift  Command,  Strategic  Air  Command,  and  Tactical  Air  Command  training 
sites.  The  primary  purpose  of  the  survey  was  to  provide  a  database  that  could  be  useo  in 
defining  the  requirements  for  ATD  procurements  and  in  developing  future  ATD  training  programs. 
In  general,  the  features  that  were  rated  highest  were  those  useo  for  training  management, 
variation  of  task  diff icul ty/f f del i ty ,  and  monitoring  student  performance.  The  level  of  AIF  use 
was  affected  somewhat  by  hardware  ano/or  software  unreliability,  implementation  time,  functional 
limitations,  and  design  deficiencies.  However,  the  presumed  training  value  of  an  AIF  was  the 
most  important  determinant  of  its  use.  Recommendations  are  made  concerning  the  AIF  capabilities 
of  future  ATDs  and  research  aimed  at  determining  the  principles  of  effective  AIF  use. 
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Cheal,  M. ,  Lyon,  D.,  &  Hubbard,  O.C.  ~  Visual  attention  effects  on  discrimination  of  line 
orientation  and  line  arrangement  (AFHRL-TR-87-22).  Williams  AFB,  A2:  Operations  Training 
Di vision. 

In  looking  at  characters  on  a  page,  some  characters  seem  to  “pop  out."  One  example  is  a 
slanted  line  among  a  group  of  vertical  lines.  It  has  been  assumed  that  this  phenomenon  occurs 
because  the  response  to  such  stimuli  Is  automatic  or  preattentlve.  If  this  is  true,  does  focal 
attention  have  an  effect  on  discrimination  of  such  stimuli?  In  the  first  experiment,  attention 
was  directed  to  characters  In  peripheral  vision  while  the  eyes  remained  stationary.  It  was  found 
that  discrimination  of  the  direction  of  a  slanted  line  was  minimally  facilitated  by  time  to  shift 
attention  to  the  target,  whereas  discrimination  of  the  direction  of  a  target  composed  of  two  line 
segments  (a  sideways  T)  required  time  to  shift  and  focus  attention  and  benefited  from  longer 
periods  that  allowed  attention  to  accumulate  at  the  target.  In  a  second  experiment,  it  was  shown 
that  accuracy  was  much  poorer  If  attention  was  misdirected  to  a  nontarget  area,  for  both 
discrimination  of  slanted  lines  and  discrimination  of  Ts.  Differences  In  the  effect  of  attention 
on  discrimination  of  the  two  types  of  stimuli  may  occur  because  only  discrimination  of  Ts 
requires  focus  of  attention  on  the  target.  On  the  other  hand,  both  SLANTS  and  Ts  may  be  affected 
by  removal  of  attention  from  an  Incorrectly  cued  location. 


Collins,  D.L.,  Hernandez,  J.M.,  Ruck,  H.W.,  Vaughan,  D.S.,  Mitchell,  J.L.,  4  Rueter,  F.H. 
Training  Decisions  System:  Overview,  design,  and  data  requirements^  ( AFHRL-TP-87-2b) .  Brooks 
AFB,  TX:  Training  Systems  Division. 


The  Training  Decisions  System  (TDS)  is  being  developed  to  provide  a  more  unified  and 
integrated  approach  to  training  programming  and  planning.  Specifically,  TDS  uses  information 
obtained  from  job  tasks  performed  by  airmen,  combined  with  airman  assignment  information  and  Air 
Force  training  capacities,  to  determine  what  cost-effective  training  options  are  available. 

Recent  Air  Force  budgetary  constraints  have  resulteo  in  a  reduced  supply  of  money  and 
personnel  for  accomplishing  Air  Force  training.  Consequently,  training  decisions  in  the  Air 
Force  are  becoming  increasingly  critical.  Furthermore,  making  those  decisions  has  been  made  more 
complex  by  incomplete  and  inadequate  cost  data.  Due  to  the  scope  and  complexity  of  Air  Force 
training,  the  challenge  has  been  in  deciding  what  to  train  (training  content),  where  to  train 

(appropriate  training  settings),  and  when  training  should  occur  (at  what  point  in  an  airman's 

career).  The  resulting  decisions,  although  the  best  possible  given  the  information  available  at 
the  time,  could  have  benefited  from  a  large,  accurate  data  base  to  help  answer  these  questions. 

TDS  is  an  extensive  multi-year  research  and  development  effort  consisting  of  three  basic 

subsystems  and  a  fourth  Integrating  subsystem.  The  first  subsystem,  the  Task  Characteristics 

Subsystem,  will  provide  task  training  modules  ano  associated  training  site  allocation  preference 
data.  These  modules  will  be  the  prime  building  blocks  for  the  other  subsystems.  The  second 
subsystem,  the  Field  Utilization  Subsystem,  will  provide  present  and  alternative  training/ 

personnel  assignment  patterns  and  associated  preference  values.  These  training/personnel 

assignment  patterns  will  be  the  prime  focus  of  analysis  for  the  TDS.  The  third  subsystem,  the 
Resource/Cost  Subsystem,  will  provide  costs  and  capacity  indicators  for  each  task  module  for  each 
training  site.  These  costs  and  capacity  functions  will  be  an  important  input  to  the  training 
optimization  routines  contained  in  the  Integration  and  Optimization  Subsystem.  This  subsystem 

will  result  in  the  integration  of  the  three  previously  described  subsystems.  In  addition,  it 
will  provide  optimization  software  and  an  interactive  system  that  will  allow  managers  to  answer 
"what  if"  questions  relative  to  training. 


Geri,  G.A. ,  4  Hubbard,  D.C.  “Visual  contrast  sensitivity  functions:  A  comparison  of  two 
psychophysical  procedures  using  untrained  observers,  (AFHRL-TR-87-26) .  Williams  AFB,  A Z: 
Operations  Training  Division. 

The  contrast  sensitivity  function  (CSF)  is  rapidly  replacing  tests  of  single  visual  acuity  as 
a  criterion  for  establishing  visual  capability.  It  has  been  shown,  for  Instance,  that 
individuals  with  the  same  visual  acuity  may  have  drastically  different  CSFs,  and  that  those 
differences  may  predict  differences  In  performance  on  various  visual  tasks.  The  criteria  for 
deciding  among  the  techniques  available  for  obtaining  CSFs  is  usually  some  measure  of  efficiency 
or  statistical  variability.  There  have  been  relatively  few  attempts,  however,  at  establishing 
practical  criteria  relevant  to  a  given  task.  The  present  experiment  is  concerned  with  obtaining 
valid  CSFs  from  untrained  jj£servers.  Practical  considerations  dictated  that  the  testing  should 
last  no  longer  than  30  minutes  and  that  the  procedures  should  be  simple  enough  for  untrained 
observers  to  perform.  Toward  this  end,  two  standard  psychophysical  procedures  were  evaluated:  a 
tracking  method  in  which  .stimulus  onset  was  graoual  and  a  "yes-no”  staircase  method  in  which  the 
stimulus  was  flashed  for/ 0.25  second.  Both  procedures  resulted  in  repeatable  CSFs  across  days; 
but  the  tracking  procedure  could  be  performed  in  less  time  and  was  subjectively  easier,  as 
determined  by  the  observers  tested.  No  statistically  significant  differences  were  found  between 
the  two  procedures. 


Lyon,  D.R.  How  quickly  can  attention  affect  form  perception?  (AFHRL-TR-87-28) .  Williams  AFB, 
AZ:  Operations  Training  Division. 

This  research  examines  the  phenomenon  of  internal  attention  shifting;  that  is,  paying 
attention  to  different  places  in  the  visual  field  without  changing  the  direction  that  the  eyes 
are  pointing.  Recently,  some  researchers  have  suggested  that  sequences  of  internal  attention 
shifts  may  De  a  necessary  part  of  visual  perception.  This  seems  to  require  that  attention  be 
snir'tea  very  rapidly,  since  a  complex  scene  can  often  be  seen  with  only  brief  presentation.  The 
purpose  of  tne  experiments  in  this  report  was  to  find  out  how  fast  such  attention  shifts  are. 
The  results  indicated  that  it  is  possible  to  shift  attention  from  one  visual  location  to  another 
in  less  than  68  milliseconds,  but  that  vision  continues  to  improve  for  120-150  milliseconds  after 
a  cue  to  shift  attention.  Furthermore,  it  was  shown  that  attention  increases  the  rate  at  which 
information  is  extracted  from  a  stimulus.  Thus,  internal  attention  shifts  are  fast  enough  to  be 
used  in  normal  visual  perception.  They  may  also  be  a  component  of  skilled  performance  in 
vision-dependent  tasks. 


Stephenson,  R.W.,  4  Gentner,  F.C.  Manpower,  personnel,  training,  and  safety  guidance  and  control 
for  weapon  system  acquisitions  .(AFHRL-TP-87-31 ).  Brooks  AFB,  TX:  Special  Projects  Office. 

The  need  for  manpower,  personnel,  training,  and  safely  (MPTAS)  guidelines  and  constraints  can 
originate  at  both  the  specific  weapon  system  and  aggregate  system  level s— whereas  the  typical 
Government  acquisition  team  specializes  only  in  information  at  the  first  (weapon  system  design) 
level.  The  amount  of  organizational  support  provided  them  is  also  not  adequate  to  their  task. 
In  order  to  help  integrate  MPTAS  factors  during  weapon  system  acquisitions,  the  Government 
needs:  (a)  enhanced  analytic  capabilities  to  analyze  total  system  tradeoffs  between  man  and 

machine  in  the  performance,  maintenance,  and  support  of  system  tasks;  (b)  interactive 
communications  with  experts  in  system  utilization  policy  and  aggregate  system  constraints;  (c) 
MPTAS-oriented  Incentive  systems  for  Government,  as  well  as  for  contractor  personnel;  and  (d)  a 
strong  centralized  headquarters  advocate  for  MPT4S  factors  with  the  authority  to  establish 
policies  and  procedures  for  acceptable  MPT4S  guidance  and  control.  Specific  control  guidance  is 
also  needed  by  Government  acquisition  teams  and  teams  of  contractor  personnel.  For  this  purpose, 
recent  case  studies  of  Government  guidance  and  control  were  analyzed,  and  two  lists  of  “do's"  and 
"don'ts"  were  developed. 
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